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Abstract: The term colorimetric imaging 15 used to describe the process of recording an image 
where the pixels represent quantitatively accurate color information. This talk will give the 
background for colorimetric imaging, why it is important, the hardware and software currently in 
use and the research problems that are of interest. Accurate color recording requires much more 
than three channels of red, green and blue. Although the eye has only three bands of color sensors 
in the cones of the retina, this does not mean that an instrument needs only three channels to 
measure color. Color is more complex and is the result of a continuous spectrum of energy from 
about 400 nm to 700 nm in wavelength. To accurately record the color of an object, we need to 
sample this spectrum with sufficient resolution to reproduce the effects of lighting on the 
appearance of the scene or object. In addition to maintaining the accurate appearance of objects, 
the accurate recording of the spectrum is required for many applications, ranging from medical 
diagnoses to pattern recognition and tracking to quality control of textile and food processing. 


Keywords: Color Imaging, Accurate Color. 

References: 

1. H. J. Trussell, D. Z. Baron, “Creating Analytic Online Homework for Digital Signal 
Processing." IEEE Signal Processing Magazine, Vol. 32, No. 5 pp. 112-118, Sept. 2015 


2. R. Cao, H. J. Trussell, R. Shamey, "Comparison of the performance of inverse transformation 
methods from OSA-UCS to CIEXYZ," JOSA A, Vol. 30, Issue 8, pp. 1508-1515, Aug. 2013. 


3. H. J. Trussell, A. О. Ercan, М. С. Kingsbury: “Color filters: When 'optimal' is not optimal", 
IEEE International Conf. on Image Processing, Phoenix, AZ, Sept 2016. 


4. H.O. Ozturk, A. N. Duca, D. Raubenheimer, H. J. Trussell “JITAR online modules to improve 
math preparation of engineering students: Preliminary results," 2015 ASEE Annual Conference 
and Exposition, Seattle, WA June 14-17, 2015 10.18260/p.24384 


5. H.J. Trussell and M. J. Vrhel, “Color Estimation Under Poisson Noise," IEEE Int. Conf. 
Acoustics, Speech and Sig. Proc., 26-31 May, 2013, Vancouver, BC, Canada. 


6. H.J. Trussell , R. Zhang, “Тһе Dominance of Poisson Noise in Digital Cameras," IEEE Int. 
Conf. Image Proc., 30 Sept.-3 Oct. 2012, Orlando, FL 


Joel Trussell received degrees from Georgia Tech (BS), Florida State (MS) 
and the University of New Mexico (PhD). He began working in the image 
and signal processing in 1971. He is currently a professor in the Electrical 
and Computer Engineering Department at North Carolina State University, 
Raleigh, NC. During 1978-79, he was a visiting professor at Heriot-Watt 
University, Edinburgh, Scotland where he worked with both the university 





| and with industry on image processing problems. During 1988-89, he was 
a visiting scientist at the Eastman Kodak Company in Rochester, NY and 1997-98 was a visiting 
scientist at Color Savvy Systems in Springboro, OH. He was a Visiting Fellow Commoner at 
Trinity College Cambridge University in 2007 and in 2015, and a visiting scientist with Hewlett- 
Packard Labs in 2008. From 2002 to 2010, he served as Director of Graduate Programs for the 
ECE Department. He is a past associate editor for the Transactions on ASSP and the Signal 
Processing Letters. He is a past chairman of and long serving on the Image and Multidimensional 
Digital Signal Processing Committee of the Signal Proceesing Society of the IEEE. He 15 currently 
Editor-in-Chief of the Proceedings of the IEEE, where he has been on the editorial board 15 2006. 
He 1s Fellow of the IEEE and has shared the IEEE-ASSP Society Senior Paper Award (1986 with 
M.R. Civanlar) and the IEEE-SP Society Paper Award (1993 with P.L. Combettes). He 15 the 
author of two texts: Fundamentals of Digital Imaging (with M. J. Vrhel) and Mathematics: The 
Language of Electrical and Computer Engineering (with Y. Viniotis). 


*' This use of this work is restricted solely for academic purposes. The author of this work owns the copy 
right and no reproduction in any form is permitted without written permission by the author. * 


NC STATE UNIVERSITY 





Color Imaging 
for 
Accurate Color 





H. J. Trussell 
Dept. of Electrical and Computer Engineering 
North Carolina State University 
Raleigh, NC 27695-7911 
hjt@ncsu.edu 


NC STATE UNIVERSITY 
What is Accurate Color? 


The quantitative description of the object so that any 
observer can determine how that object would appear 
under any illumination. 


Topics: 

е Spectral description of color 
* Recording of color 

° [Instrumentation 

* Cameras 

* Applications 

* Research problems 
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Figure 8.8. Cone sensitivities of the human eye. 
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Color Measurement Devices 








Spectroradiometer — 
measures radiant source 
(visible spectrum) 


Measures radiant spectrum 
from UV to near IH 


Applications : 

Precise colorimetry, lighting, 
LEDs, calibrating displays 
and colorimeters, chemical 
characterization, paints and 
goniometry 
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Color Measurement Devices 


Spectrophotometer — 
measures reflectance and " 
transmission (visible м, 
Spectrum) => 


Applications : 

Precise colorimetry, textiles, 
printing, paints, gloss, food, 
packaging, chemical 
processes 
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Color Image Recording Devices 


* Digital still camera 

* Digital video camera 
* Document scanner 
* Book scanners 





All are three-band recorders 
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Color Constancy: Your vision system will adjust to the lighting 
that illuminates an object, but the camera will not! 





Bottom 
For accurate color, we need to measure the reflectance Cyan 
of the object, so we can reproduce its color under any 
illumination 
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YZ Color Matching Functions 


CIE XYZ color matching functions 
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Effect of Illumination: Ее eerie dob occa 


The spectrum of the illuminant effectively changes the color matching 
functions 


PF reflectance spectrum 
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Effect of Illumination: 


Metamerism: objects look different under different illuminations 





Metamerism: “Does it match the 


ТЕ 


standard under ай illuminants? 
http://knowledgeandfashion.blogspot.com/2011/07/metamarism- 
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Cameras: Try to guess the lighting (white balance) 





httos://argspin.con/blog/ 
5 се. 





http://www. photoplusmag.cony/2012/02/25/interior- 
photography-best-canon-dsir-settings-for-shooting-indoors 
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Digital Spectral Cameras 
Applications in museum art recording 


Later printing quality control 






VASARI Scanner (1992) 
Ecole Nationale Supérieure des Télécommunications 
Filter Wheel , Scanner— 7 bands, 3000x2000 pixels 





VASARI Scanner (2000) ENST 
Filter Wheel , Scanner— 12 bands, 3000x2000 pixels 





| wa 


DX f^ MARC CCD Camera (2001) ENST 
¿ t" CFA 3 bands, 20k x 20k pixels, micropositioning 
compensate for CFA, macropositioning scans image 
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Digital Spectral Cameras 


Tricor Imaging 
Spectrophotometer 
Silicon CCD 
1280x960 

Filter wheel 
Interpolated to 380- 
/80nm; 20nm 

60 seconds 








Modified Sinar Camera 

Rochester Institute of Technology 

2 filters + 3-band CFA, 6 bands, interpolated to 
380-/30nm, 4080x5440 pixels micropositioning 
used to compensate for CFA 
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Digital Still Camera — Pipeline 


1. Exposure adjustment 





2. Optics/focus 


3. Initial image capture/digitization —— — 








4. Dark noise removal parcar _! — 


5. Uniformity adjustments 


6. Demosaicking 
7. White balance 





9. Appearance adjustments 


9. Compression/file formatting 
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Demosaicking — Bilinear 
Interpolation 
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Original- high resolution Bilinear interpolation of 
mosaicked data 
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Demosaicking Approaches 


Use correlation between bands but avoid interpolation 
across edge boundaries 


Estimate edge gradients first then use these to weight 
interpolation of adjacent bands. May use appearance 
related properties to determine the color space in which 
to interpolate. 


Use of local/global statistical characterization to enforce 
constraints or properties 
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Mosaic Patterns for Color Filter Arrays NC STATE UNIVERSITY 


(CFAs) 


Bayer Array 3 Hyperspectral Array 36 
bands bands 











Bayer Array + 
Filtered Array 
6 bands 
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Demosaicking Hyperspectral Arrays 


opectral estimation using multiple bands is well understood. 
For low spatial frequency images, the estimates will be good. 


Combining multiple bands for sub-block resolution is a problem. 


The usual strategy of edge detection using the green band is not 
applicable 


Edge detection Is still a problem that will need to be addressed for 
sub-block resolution 


otatistical characterization of the various bands and their 
correlations need to be addressed. 
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Color Filter Arrays - Sensors 


Typical Hardware Problems for CFAs 


Uniform pixel sensitivity — response of the unfiltered 
CCD/CMOS elements varies spatially 


Temporal pixel sensitivity — response of the sensor 
elements varies with temperature 


Nonuniform pixel filters — all filters for a specific band 
may not be the same, vary with spatial location 


Color filter fabrication errors, limitations 


Noise may be both spatially varying and signal 
dependent , as well as temperature dependent 
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Colorimetry for Cameras and Scanners 
Sensor Filters - they are not equivalent to either the human eye or the 


CIE XYZ color matching functions, need better control of the 
transmission spectra 


Bayer Mosaic Filter Array 
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Results of three filter measurements must be transformed by mathematical 
estimation method to approximate CIE values 
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Color Filter Arrays - Sensors 
Noise Characteristics for CFAs 
Thermal (dark current) noise - Gaussian 
Head out noise — transfer of charge in pixel to ADC 
Fixed pattern noise (e.g., hot pixels) 
Photon response non-uniformity, filter variation 


Photon shot noise (Poisson) 
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: Types of Noise 


Wariance vs. Mean Гог Red channel 
alpha = 2.9489 , beta = -6117.4072 


5000 6000 тою 


Variance vs. Мал Dien снаппе! 
alpha = 2.8099 , beta = -5705.5450 


Variance vs. aan цо спаппе! 
alpha = 2.9907 , beta = -621 5.8026 
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Fig. 6. Уапапсе vs. Mean Curves for Canon Rebel Т3 Raw Data 
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- Types of Noise 
SRGB data (after demosaicking): 






Variance vs. Mean far R channel 


Меап( ғ) 
Variance vs. Mean for Green channel 
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Меап! ғ) 
Variance vs, Mean for Blue channel 


Fig. 5. Variance vs. Mean Curves for Nikon JPEG Data 
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Color Filter Arrays - Sensors 
CCD vs CMOS sensors 
Typically — CCD sensors are less noisy, more 
sensitive to light, less variability spatially, and use 
global shutter 
CMOS sensors are cheaper, faster, more power 
efficient, better treatment of over exposure 


(blooming), and use rolling shutter — problems with 
video 
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Color Filter Arrays - Sensors 


Foveon sensor (CMOS) 


Foveon X39 Traditional 
direct image sensor CCD/CMOS sensor 
Silicon color absorption Foveon X3 sensor stack 
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Color Imaging 
High dynamic range sensing 


Record accurate information across the entire 
dynamic range of the scene 


Light intensity varies over a range of 10° 


Need to have adaptive sensitivity in light and dark 
regions — but record the relative exposure 
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Color Imaging 


High dynamic range imaging: example 





iPhone uses 3 
images to create 
an HDK таве http://en.wikipedia.org/wiki/ 
File:StLouisArchMultExpTon 
eMapped.jpg 
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Accurate Color — what's it good for? и 


Applications: 


° improved color recording of natural scenes 
* Fasier archiving of museum art 
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Accurate Color — what’s it good for? ШО 


Applications: 


° Improved documentation and diagnosis of medical 
conditions that depend on color, e.g., dermatology, 
ophthomology, endoscopy 
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Accurate Color — what's it good for? | = 


Applications: 


* Color quality control of complex patterns, e.g., textiles, 
printing, packaging 
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Accurate Color — what’s it good for? Å 


Applications: 


e Quality control of directional effects, e.g., matte, gloss, 
metallic paints — allows spatial statistics 
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Accurate Color — what's it good for? | | 


Applications: 


е Space and planetary exploration 


Planetary Surfaces 


Asteroid Itokawa Tita: 
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Research Problems and Questions 


Hyperspectral demosaicking 

Hyperspectral mosaic patterns 

Automatic calibration 

Fabrication of accurate color filters 

Adaptive color capture (transverse field detector) 
High dynamic range sensors and displays 


Simultaneous range and color sensors 
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